{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-title"
    ]
   },
   "source": [
    "# Dropout\n",
    "Dropout [1] is a technique for regularizing neural networks by randomly setting some output activations to zero during the forward pass. In this exercise you will implement a dropout layer and modify your fully-connected network to optionally use dropout.\n",
    "\n",
    "[1] [Geoffrey E. Hinton et al, \"Improving neural networks by preventing co-adaptation of feature detectors\", arXiv 2012](https://arxiv.org/abs/1207.0580)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": [
     "pdf-ignore"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "# As usual, a bit of setup\n",
    "from __future__ import print_function\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cs231n.classifiers.fc_net import *\n",
    "from cs231n.data_utils import get_CIFAR10_data\n",
    "from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array\n",
    "from cs231n.solver import Solver\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "def rel_error(x, y):\n",
    "  \"\"\" returns relative error \"\"\"\n",
    "  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": [
     "pdf-ignore"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train:  (49000, 3, 32, 32)\n",
      "y_train:  (49000,)\n",
      "X_val:  (1000, 3, 32, 32)\n",
      "y_val:  (1000,)\n",
      "X_test:  (1000, 3, 32, 32)\n",
      "y_test:  (1000,)\n"
     ]
    }
   ],
   "source": [
    "# Load the (preprocessed) CIFAR10 data.\n",
    "\n",
    "data = get_CIFAR10_data()\n",
    "for k, v in data.items():\n",
    "  print('%s: ' % k, v.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout forward pass\n",
    "In the file `cs231n/layers.py`, implement the forward pass for dropout. Since dropout behaves differently during training and testing, make sure to implement the operation for both modes.\n",
    "\n",
    "Once you have done so, run the cell below to test your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests with p =  0.25\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  10.014059116977283\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.749784\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.4\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  9.977917658761159\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.600796\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.7\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  9.987811912159426\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.30074\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(500, 500) + 10\n",
    "\n",
    "\n",
    "for p in [0.25, 0.4, 0.7]:\n",
    "  out, _ = dropout_forward(x, {'mode': 'train', 'p': p})\n",
    "  out_test, _ = dropout_forward(x, {'mode': 'test', 'p': p})\n",
    "\n",
    "  print('Running tests with p = ', p)\n",
    "  print('Mean of input: ', x.mean())\n",
    "  print('Mean of train-time output: ', out.mean())\n",
    "  print('Mean of test-time output: ', out_test.mean())\n",
    "  print('Fraction of train-time output set to zero: ', (out == 0).mean())\n",
    "  print('Fraction of test-time output set to zero: ', (out_test == 0).mean())\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout backward pass\n",
    "In the file `cs231n/layers.py`, implement the backward pass for dropout. After doing so, run the following cell to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  1.892896954038074e-11\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(10, 10) + 10\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "dropout_param = {'mode': 'train', 'p': 0.2, 'seed': 123}\n",
    "out, cache = dropout_forward(x, dropout_param)\n",
    "dx = dropout_backward(dout, cache)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_forward(xx, dropout_param)[0], x, dout)\n",
    "\n",
    "# Error should be around e-10 or less\n",
    "print('dx relative error: ', rel_error(dx, dx_num))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 1:\n",
    "What happens if we do not divide the values being passed through inverse dropout by `p` in the dropout layer? Why does that happen?\n",
    "\n",
    "## Answer:\n",
    "p是保留神经元的概率，不除以p，则训练与测试时候的scale不同\n",
    "会造成测试的scale，\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fully-connected nets with Dropout\n",
    "In the file `cs231n/classifiers/fc_net.py`, modify your implementation to use dropout. Specifically, if the constructor of the network receives a value that is not 1 for the `dropout` parameter, then the net should add a dropout layer immediately after every ReLU nonlinearity. After doing so, run the following to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running check with dropout =  1\n",
      "Initial loss:  2.3004790897684924\n",
      "W1 relative error: 1.48e-07\n",
      "W2 relative error: 2.21e-05\n",
      "W3 relative error: 3.53e-07\n",
      "b1 relative error: 5.38e-09\n",
      "b2 relative error: 2.09e-09\n",
      "b3 relative error: 5.80e-11\n",
      "\n",
      "Running check with dropout =  0.75\n",
      "Initial loss:  2.302454105929313\n",
      "W1 relative error: 2.56e-07\n",
      "W2 relative error: 2.10e-06\n",
      "W3 relative error: 5.57e-08\n",
      "b1 relative error: 6.43e-09\n",
      "b2 relative error: 2.04e-09\n",
      "b3 relative error: 1.10e-10\n",
      "\n",
      "Running check with dropout =  0.5\n",
      "Initial loss:  2.3030067920900876\n",
      "W1 relative error: 1.22e-06\n",
      "W2 relative error: 9.41e-08\n",
      "W3 relative error: 2.67e-07\n",
      "b1 relative error: 3.53e-08\n",
      "b2 relative error: 3.37e-09\n",
      "b3 relative error: 1.01e-10\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "N, D, H1, H2, C = 2, 15, 20, 30, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "\n",
    "for dropout in [1, 0.75, 0.5]:\n",
    "  print('Running check with dropout = ', dropout)\n",
    "  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,\n",
    "                            weight_scale=5e-2, dtype=np.float64,\n",
    "                            dropout=dropout, seed=123)\n",
    "\n",
    "  loss, grads = model.loss(X, y)\n",
    "  print('Initial loss: ', loss)\n",
    "  \n",
    "  # Relative errors should be around e-6 or less; Note that it's fine\n",
    "  # if for dropout=1 you have W2 error be on the order of e-5.\n",
    "  for name in sorted(grads):\n",
    "    f = lambda _: model.loss(X, y)[0]\n",
    "    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)\n",
    "    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularization experiment\n",
    "As an experiment, we will train a pair of two-layer networks on 500 training examples: one will use no dropout, and one will use a keep probability of 0.25. We will then visualize the training and validation accuracies of the two networks over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "(Iteration 1 / 125) loss: 7.856643\n",
      "(Epoch 0 / 25) train acc: 0.260000; val_acc: 0.184000\n",
      "(Epoch 1 / 25) train acc: 0.416000; val_acc: 0.258000\n",
      "(Epoch 2 / 25) train acc: 0.482000; val_acc: 0.276000\n",
      "(Epoch 3 / 25) train acc: 0.532000; val_acc: 0.277000\n",
      "(Epoch 4 / 25) train acc: 0.600000; val_acc: 0.271000\n",
      "(Epoch 5 / 25) train acc: 0.708000; val_acc: 0.299000\n",
      "(Epoch 6 / 25) train acc: 0.722000; val_acc: 0.282000\n",
      "(Epoch 7 / 25) train acc: 0.832000; val_acc: 0.255000\n",
      "(Epoch 8 / 25) train acc: 0.878000; val_acc: 0.269000\n",
      "(Epoch 9 / 25) train acc: 0.902000; val_acc: 0.275000\n",
      "(Epoch 10 / 25) train acc: 0.890000; val_acc: 0.261000\n",
      "(Epoch 11 / 25) train acc: 0.930000; val_acc: 0.282000\n",
      "(Epoch 12 / 25) train acc: 0.958000; val_acc: 0.300000\n",
      "(Epoch 13 / 25) train acc: 0.964000; val_acc: 0.305000\n",
      "(Epoch 14 / 25) train acc: 0.962000; val_acc: 0.314000\n",
      "(Epoch 15 / 25) train acc: 0.964000; val_acc: 0.304000\n",
      "(Epoch 16 / 25) train acc: 0.982000; val_acc: 0.309000\n",
      "(Epoch 17 / 25) train acc: 0.972000; val_acc: 0.323000\n",
      "(Epoch 18 / 25) train acc: 0.992000; val_acc: 0.316000\n",
      "(Epoch 19 / 25) train acc: 0.984000; val_acc: 0.303000\n",
      "(Epoch 20 / 25) train acc: 0.986000; val_acc: 0.313000\n",
      "(Iteration 101 / 125) loss: 0.011629\n",
      "(Epoch 21 / 25) train acc: 0.994000; val_acc: 0.306000\n",
      "(Epoch 22 / 25) train acc: 0.976000; val_acc: 0.312000\n",
      "(Epoch 23 / 25) train acc: 0.962000; val_acc: 0.319000\n",
      "(Epoch 24 / 25) train acc: 0.978000; val_acc: 0.306000\n",
      "(Epoch 25 / 25) train acc: 0.982000; val_acc: 0.308000\n",
      "\n",
      "0.25\n",
      "(Iteration 1 / 125) loss: 4.897379\n",
      "(Epoch 0 / 25) train acc: 0.240000; val_acc: 0.196000\n",
      "(Epoch 1 / 25) train acc: 0.388000; val_acc: 0.257000\n",
      "(Epoch 2 / 25) train acc: 0.420000; val_acc: 0.263000\n",
      "(Epoch 3 / 25) train acc: 0.516000; val_acc: 0.290000\n",
      "(Epoch 4 / 25) train acc: 0.568000; val_acc: 0.296000\n",
      "(Epoch 5 / 25) train acc: 0.620000; val_acc: 0.299000\n",
      "(Epoch 6 / 25) train acc: 0.674000; val_acc: 0.301000\n",
      "(Epoch 7 / 25) train acc: 0.692000; val_acc: 0.307000\n",
      "(Epoch 8 / 25) train acc: 0.732000; val_acc: 0.310000\n",
      "(Epoch 9 / 25) train acc: 0.758000; val_acc: 0.307000\n",
      "(Epoch 10 / 25) train acc: 0.780000; val_acc: 0.315000\n",
      "(Epoch 11 / 25) train acc: 0.788000; val_acc: 0.308000\n",
      "(Epoch 12 / 25) train acc: 0.824000; val_acc: 0.320000\n",
      "(Epoch 13 / 25) train acc: 0.854000; val_acc: 0.334000\n",
      "(Epoch 14 / 25) train acc: 0.850000; val_acc: 0.335000\n",
      "(Epoch 15 / 25) train acc: 0.880000; val_acc: 0.323000\n",
      "(Epoch 16 / 25) train acc: 0.876000; val_acc: 0.303000\n",
      "(Epoch 17 / 25) train acc: 0.898000; val_acc: 0.312000\n",
      "(Epoch 18 / 25) train acc: 0.908000; val_acc: 0.332000\n",
      "(Epoch 19 / 25) train acc: 0.924000; val_acc: 0.348000\n",
      "(Epoch 20 / 25) train acc: 0.924000; val_acc: 0.332000\n",
      "(Iteration 101 / 125) loss: 1.076584\n",
      "(Epoch 21 / 25) train acc: 0.904000; val_acc: 0.322000\n",
      "(Epoch 22 / 25) train acc: 0.912000; val_acc: 0.312000\n",
      "(Epoch 23 / 25) train acc: 0.918000; val_acc: 0.306000\n",
      "(Epoch 24 / 25) train acc: 0.908000; val_acc: 0.307000\n",
      "(Epoch 25 / 25) train acc: 0.924000; val_acc: 0.321000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "np.random.seed(231)\n",
    "num_train = 500\n",
    "small_data = {\n",
    "  'X_train': data['X_train'][:num_train],\n",
    "  'y_train': data['y_train'][:num_train],\n",
    "  'X_val': data['X_val'],\n",
    "  'y_val': data['y_val'],\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_choices = [1, 0.25]\n",
    "for dropout in dropout_choices:\n",
    "  model = FullyConnectedNet([500], dropout=dropout)\n",
    "  print(dropout)\n",
    "\n",
    "  solver = Solver(model, small_data,\n",
    "                  num_epochs=25, batch_size=100,\n",
    "                  update_rule='adam',\n",
    "                  optim_config={\n",
    "                    'learning_rate': 5e-4,\n",
    "                  },\n",
    "                  verbose=True, print_every=100)\n",
    "  solver.train()\n",
    "  solvers[dropout] = solver\n",
    "  print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4UAAAJNCAYAAACGHNDNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdf5hcZX3//+fbsMgKmOVHRLIbTBQMKkRCl6BCaWs+GlCRgG0EWyhYRVCL7bfNR+h1FSn99isaLZZCS7Eg0lJpWiGCgtEGqkJVWAgm/DASEcluqMRgIuIiIby/f8xsnCyTsJvszJmZ83xcV66Zc58zc96TPbvZV+773HdkJpIkSZKkcnpR0QVIkiRJkopjKJQkSZKkEjMUSpIkSVKJGQolSZIkqcQMhZIkSZJUYoZCSZIkSSoxQ6EkqfQi4paI+MOi65AkqQjhOoWSpHYUEb+o2XwJ8Ctgc3X7A5l5bfOrkiSp/RgKJUltLyIeAd6Xmf9VZ98umfls86tqvIgIKv+WP1d0LZKk9uXwUUlSR4mI346IwYj4aET8L/C5iNgrIr4cEesi4mfV5301r/nviHhf9fnpEXF7RHyqeuyPIuK47Zzv3Ij4YUQ8GREPRMSJo/a/PyIerNl/eLV9WkRcX61pfURcWm2/ICL+teb10yMiI2KXmlr/JiLuAH4JvDIizqg5x8MR8YFRNZwQEfdGxM+rtR4bEb8XEXePOu7PImLJDv7VS5LalKFQktSJXg7sDbwCOJPKv3efq24fAAwDl27n9UcCq4B9gU8CV1Z75er5IfCbwGTgr4B/jYj9ASLi94ALgNOAlwLvBNZHxCTgy8CPgelAL3DdOD7fqdXPtWf1PR4H3lE9xxnAxTXhcw5wDbAQ6AGOAR4BbgRmRMRrat73D4B/GUcdkqQOYCiUJHWi54CPZeavMnM4M9dn5hcz85eZ+STwN8Bvbef1P87Mz2bmZuDzwP7AfvUOzMz/yMy1mflcZv478BAwp7r7fcAnM/OurFidmT+u7p8KLMzMpzLz6cy8fRyf7+rMvD8zn83MTZn5lcz8YfUc3wC+RiWoAvwRcFVmfr1a41Bmfj8zfwX8O5UgSES8jkpA/fI46pAkdQBDoSSpE63LzKdHNiLiJRHxTxHx44j4OfBNoKfaY1fP/448ycxfVp/uUe/AiDitOjRzQ0RsAA6h0sMIMI1KT+Jo06gEzx2913HNqBqOi4jvRMQT1RreNoYaoBJ431PtBT0VWFwNi5KkEjEUSpI60ehZ1P4MmAkcmZkvpTKEEmBbQ0LHJCJeAXwW+DCwT2b2APfVvO8a4FV1XroGOGDkPsFRnqIym+qIl9c5Zsvni4gXA18EPgXsV63h5jHUQGZ+B3iGSq/ie3DoqCSVkqFQklQGe1K5j3BDROwNfGyC3nd3KgFtHUBEnEGlp3DEPwN/HhG/ERUHVoPkncBjwEURsXtE7BYRR1Vfcy9wTEQcEBGTgfNeoIZdgRdXa3i2OinOW2v2XwmcERFzI+JFEdEbEQfX7L+Gyv2Vz45zCKskqUMYCiVJZfAZoBv4KfAd4KsT8aaZ+QDwaeDbwE+AQ4E7avb/B5X7F/8NeBJYAuxdvVfxeOBA4FFgEHh39TVfp3Kv3wrgbl7gHr/qPZLnAIuBn1Hp8buxZv+dVCefATYC36Ay4c6If6ESZO0llKSScp1CSZJKLCK6qcxeenhmPlR0PZKk5rOnUJKkcjsbuMtAKEnlVe8Gd0mSVAIR8QiVCWnmF1yKJKlADh+VJEmSpBJz+KgkSZIklZihUJIkSZJKrBT3FO677745ffr0osuQJEmSpELcfffdP83MKfX2lSIUTp8+nYGBgaLLkCRJkqRCRMSPt7XP4aOSJEmSVGKGQkmSJEkqMUOhJEmSJJVYS4XCiLgqIh6PiPu2sT8i4pKIWB0RKyLi8GbXKEmSJEmdpKVCIXA1cOx29h8HHFT9cybwj02oSZIkSZI6VkuFwsz8JvDEdg45AbgmK74D9ETE/s2pTpIkSZI6T7stSdELrKnZHqy2PVZMOZIkSdL2LVk+xKKlq1i7YZipPd0snDeT+bN7iy5L2qLdQmHUacu6B0acSWWIKQcccEAja5IkSWo7BpXmWLJ8iPOuX8nwps0ADG0Y5rzrVwL4962W0VLDR8dgEJhWs90HrK13YGZekZn9mdk/ZcqUphQnSZLUDkaCytCGYZJfB5Uly4eKLq2hliwf4qiLbmXGuV/hqItubcrnXbR01ZZAOGJ402YWLV3V8HNLY9VuofBG4LTqLKRvADZmpkNHJUmSxqHIoFJEMBs5bxFBeO2G4XG1S0VoqeGjEfEF4LeBfSNiEPgY0AWQmZcDNwNvA1YDvwTOKKZSSZLUSco2lLKooFLkUMrtBeFGnntqTzdDdf5ep/Z0N+ycKkY7/xxpqVCYmae8wP4EPtSkciRJUgmU8Z6vooJKUcEMigvCC+fN3Or6AujumsTCeTMbet4R7RxUdkRRn7fdf4602/BRSZKkCVXGoZQL582ku2vSVm3NCCpFDqXcVuBtdBCeP7uXj590KL093QTQ29PNx086tKlBpSz3jhb5edv93tGW6imUJElqtjIOpRx5/2b3qBQ5lLLIHrv5s3sL6S0qsme2CGXsiZ4ohkJJklRqZRxKCcUElaKDGTQ/CBep3YPKeBXdE93O944aCiVJalFF3htT1C/ORZy7qKBStl/YofhgVlSPXVGKDCpFfC+XtSd6IhgKJUnajrJNWlDkkMaizl3GoZRFKlswK1JRQaWo72V7ondcVCb07Gz9/f05MDBQdBmSpDYz+hcbqPyC0YxJIo666Na6gaG3p5s7zn1zx5236HMXocjrS+VRxH9sFfm9XLbZVscjIu7OzP56++wplCRpG8o4aUGRQxrLNpyy3XsW1B6K6Jkt8nvZnugdYyiUJGkbyjhpQZFDGss4nNJfYNWJyvi93O5cp1CSpG0oal0zKG4duaLOW/S5JU0cv5fbj6FQkqRtKPIXm6IWvC5yoe0izy1p4vi93H6caEaSpO1w0gJJUidwohlJknaQ93xJkjqdw0clSZIkqcTsKZQktQWHcUqS1Bj2FEqSWt7IIt9DG4ZJYGjDMOddv5Ily4eKLk3SeK1YDBcfAhf0VB5XLC7HudX52vj6MhRKklre9haRl9RGViyGm86BjWuArDzedE5zfnku8tzqfG1+fRkKJUktr8hF5CVNoGUXwqZR37ebhivtnXzuMmrjXrMd0ubXl6FQktTyilxEXtIE2jg4vvZOOXfZtHmv2Q5p8+vLUChJanlFLiIvaQJN7htfe6ecu0hF9Ni1ea/ZDmnz68tQKElqefNn9/Lxkw6lt6ebAHp7uvn4SYc6+6i0M4oIC3PPh65RPfxd3ZX2Tj53UYrqsSuy16yoYattfn213JIUEXEs8HfAJOCfM/OiUfv3Aq4CXgU8Dbw3M+9reqGSpKZyEXlpAo2EhZHenJGwADBrQePOO/Leyy6sBITJfZVfmht5zlY494rFxZx3ez12jTz/5L5qEK3T3khFXde171/E13kCRGYWXcMWETEJ+AHwFmAQuAs4JTMfqDlmEfCLzPyriDgYuCwz527vffv7+3NgYKCBlUtSc7lmn6SdcvEh2/ilfRr8qf/XPqFGBxWo9CAdf0njA8MFPUC93/UDLtjQuPMW9Zm9rrcrIu7OzP56+1pt+OgcYHVmPpyZzwDXASeMOua1wDKAzPw+MD0i9mtumZJUHNfskzpIUUPd2nxSjLZS5P11Rd3nNmtBJQBOngZE5bEZIdjreoe1WijsBWrj/WC1rdb3gJMAImIO8AqgPe7glKQJ4Jp9UococobGNp8Uo60UGVSKvM9t1oJK79wFGyqPzRhG6XW9w1otFEadttF93hcBe0XEvcAfA8uBZ5/3RhFnRsRARAysW7du4iuVpIIUuWbfkuVDHHXRrcw49yscddGt9k5KO6PIHqQ2nxSjrRQZVIrqsSuK1/UOa7WJZgaBaTXbfcDa2gMy8+fAGQAREcCPqn8YddwVwBVQuaewQfVKUtNN7elmqE4AbPSafSPDVkd6KUeGrQLezyjtiCJ7kNp8Uoy2Mvf8+vfXNSuozFpQnq+r1/UOa7VQeBdwUETMAIaAk4H31B4QET3AL6v3HL4P+GY1KEpSKSycN3OrcAbNWbNve8NWDYXSDihqhsYRZQoLRTKoNJfX9Q5pqVCYmc9GxIeBpVSWpLgqM++PiLOq+y8HXgNcExGbgQeAPyqsYEkqwEgAa/bso0UOW1WJFDV1fxGK7kFS8xhU1OJaKhQCZObNwM2j2i6vef5t4KBm1yVJoxW5LEQRa/YVNWxVJVLkGmNFhFF7kCS1iJYLhZLUDsp4f11Rw1ZVIkUttF30gteGQEkFa7XZRyWpLZRxWYj5s3v5+EmH0tvTTQC9Pd18/KRDOzYEqwBFTbxS5CygktQC7CmUpB1Q1vvrihi2qhIpauIVF7yWVHL2FErSDtjWfXTeXyfthKLWGHPBa0klZyiUpB2wcN5MursmbdXm/XXSTipqoW0XvJZUcg4flaQdUNSyEFLHK2LiFWcBlVRykZlF19Bw/f39OTAwUHQZkiSNT5nW7JMkNVRE3J2Z/fX22VMoSVIrKtuafZKkwnhPoSRJraioZRJGwujGNUD+OoyuWNzY80qSCmMolCSpFblmnySpSQyFkiS1oqKWSXDNPkkqHUOhJEmtyDX7JElNYiiU1PaWLB/iqItuZca5X+Goi25lyfKhokuSdp5r9kmSmsTZRyW1tSXLhzjv+pUMb9oMwNCGYc67fiWAawaq/blmnySpCQyFktraoqWrtgTCEcObNrNo6SpDoSZGGZdnKCKMSpIKYyiU1NbWbhgeV7s0LkWuFShJUpN4T6Gktja1p3tc7dK4uDyDJKkEDIWS2trCeTPp7pq0VVt31yQWzptZUEXqKC7PIEkqAYePSmprI/cNLlq6irUbhpna083CeTO9n1ATY3JfZchovXZJkjqEoVBS25s/u9cQqMaYe/7W9xSCyzNIkjpOyw0fjYhjI2JVRKyOiHPr7J8cETdFxPci4v6IOKOIOiVJJVDUWoGSJDVRS/UURsQk4DLgLcAgcFdE3JiZD9Qc9iHggcw8PiKmAKsi4trMfKaAkiVJnc7lGSRJHa7VegrnAKsz8+FqyLsOOGHUMQnsGREB7AE8ATzb3DIlSZIkqTO0WijsBWrv6B+sttW6FHgNsBZYCXwkM59rTnmSJEmS1FlaLRRGnbYctT0PuBeYChwGXBoRL33eG0WcGREDETGwbt26ia9UkiRJkjpAq4XCQWBazXYflR7BWmcA12fFauBHwMGj3ygzr8jM/szsnzJlSsMKliQ1yYrFcPEhcEFP5XHF4qIrkiSpI7RaKLwLOCgiZkTErsDJwI2jjnkUmAsQEfsBM4GHm1qlJKm5ViyuLA2xcQ2QlcebzjEYSpI0AVoqFGbms8CHgaXAg8DizLw/Is6KiLOqh/018KaIWAksAz6amT8tpmJJKpmieuuWXbj1WoFQ2V52YXPOL0lSB2upJSkAMvNm4OZRbZfXPF8LvLXZdUlS6Y301o2Es5HeOmj8kg0bB8fXLkmSxqylegolSS2syN66yX3ja5ckSWNmKJQkjU2RvXVzz4eu7q3buror7ZIkaacYCiVJY1Nkb92sBXD8JTB5GhCVx+MvafywVUmSSqDl7imUJLWouedvfU8hNLe3btYCQ6AkSQ1gT6EktaMiZgG1t06SpI5kT6EktZsiZwG1t06SpI5jT6EktRvX7JMkSRPIUChJ7cY1+yRJ0gQyFEpSu3HNPkmSNIEMhZLUblyzT5IkTSBDoSS1G2cBlSRJE8jZRyWpHTkLqCRJmiD2FErSjipirUBJkqQJZk+hJO2IItcKlCRJmkD2FErSjnCtQEmS1CHsKZQ0YZYsH2LR0lWs3TDM1J5uFs6byfzZvUWX1RiuFShJkjqEPYWSJsSS5UOcd/1KhjYMk8DQhmHOu34lS5YPFV1aY7hWoCRJ6hCGQkkTYtHSVQxv2rxV2/CmzSxauqqgihrMtQIlSVKHMBRKmhBrNwyPq73tuVagJEnqEN5TKHWYou7rm9rTzVCdADi1p7vO0R3CtQIlSVIHaLmewog4NiJWRcTqiDi3zv6FEXFv9c99EbE5IvYuolap1RR5X9/CeTPp7pq0VVt31yQWzpvZ8HNLkiRpx7VUKIyIScBlwHHAa4FTIuK1tcdk5qLMPCwzDwPOA76RmU80v1qp9RR5X9/82b18/KRD6e3pJoDenm4+ftKhnTv7qCRJUodoteGjc4DVmfkwQERcB5wAPLCN408BvtCk2qSWV/R9ffNn9xoCJUmS2kxL9RQCvcCamu3BatvzRMRLgGOBLzahLqktbOv+vY6+rw9gxWK4+BC4oKfyuGJx0RVJkiS1jVYLhVGnLbdx7PHAHdsaOhoRZ0bEQEQMrFu3bsIKlFpZKe/rW7EYbjoHNq4BsvJ40zkGQ0mSpDFqtVA4CEyr2e4D1m7j2JPZztDRzLwiM/szs3/KlCkTWKLUukp5X9+yC2HTqOGxm4Yr7ZIkSXpBrXZP4V3AQRExAxiiEvzeM/qgiJgM/BbwB80tT2p9pbuvb+Pg+NolSZK0lYb0FEbEOyJi3O+dmc8CHwaWAg8CizPz/og4KyLOqjn0ROBrmfnUxFQsqW1N7htfuyRJkrbSqOGjJwMPRcQnI+I143lhZt6cma/OzFdl5t9U2y7PzMtrjrk6M0+e4JoltaO550PXqIl0uror7ZIkSXpBDQmFmfkHwGzgh8DnIuLb1Ylf9mzE+SS1iCJmAZ21AI6/BCZPA6LyePwllXZJkiS9oIbdU5iZP4+ILwLdwJ9QGfK5MCIuycy/b9R5JRVkZBbQkUlfRmYBhcYHtFkLDIGSJEk7qFH3FB4fETcAtwJdwJzMPA54PfDnjTinpII5C6gkSVJbalRP4e8BF2fmN2sbM/OXEfHeBp1TUpGcBVSSJKktNWqimY8Bd45sRER3REwHyMxlDTqnpCI5C6gkSVJbalQo/A/guZrtzdU2SZ3KWUAlSZLaUqNC4S6Z+czIRvX5rg06l6RW4CygkiRJbalR9xSui4h3ZuaNABFxAvDTBp1LUqtwFlBJkqS206hQeBZwbURcCgSwBjitQeeSJEmSJO2ghoTCzPwh8IaI2AOIzHyyEeeRJEmSJO2chi1eHxFvB14H7BYRAGSmC5ZJkiRJUgtp1OL1lwPvBv6YyvDR3wNe0YhzSZIkSZJ2XKNmH31TZp4G/Cwz/wp4IzCtQeeSJEmSJO2gRoXCp6uPv4yIqcAmYEaDziVJkiRJ2kGNuqfwpojoARYB9wAJfLZB55IkSZIk7aAJD4UR8SJgWWZuAL4YEV8GdsvMjRN9Lkl1rFgMyy6EjYMwuQ/mnu/agZIkSdqmCR8+mpnPAZ+u2f6VgVBqkhWL4aZzYOMaICuPN51TaZckSZLqaNQ9hV+LiHfFyFoUkppj2YWwaXjrtk3DlXZJkiSpjkbdU/j/ALsDz0bE01SWpcjMfGmDzicJKkNGx9MuSZKk0mtIKMzMPRvxvpJewOS+6tDROu2SJElSHY1avP6Yen/G+NpjI2JVRKyOiHO3ccxvR8S9EXF/RHxjYquX2tjc86Gre+u2ru5KuyRJklRHo4aPLqx5vhswB7gbePP2XhQRk4DLgLcAg8BdEXFjZj5Qc0wP8A/AsZn5aES8bKKLlybCkuVDLFq6irUbhpna083CeTOZP7u3sScdmWXU2UclSZI0Ro0aPnp87XZETAM+OYaXzgFWZ+bD1dddB5wAPFBzzHuA6zPz0eq5Hp+QoqUJtGT5EOddv5LhTZsBGNowzHnXrwRoTjA0BEqSJGmMGjX76GiDwCFjOK4XqL0harDaVuvVwF4R8d8RcXdEnDZBNUoTZtHSVVsC4YjhTZtZtHRVQRVJkiRJ9TWkpzAi/h7I6uaLgMOA743lpXXactT2LsBvAHOBbuDbEfGdzPzBqBrOBM4EOOCAA8ZevDQB1m4YHle7JEmSVJRG3VM4UPP8WeALmXnHGF43CEyr2e4D1tY55qeZ+RTwVER8E3g9sFUozMwrgCsA+vv7RwdLqaGm9nQzVCcATu3prnO0JEmSVJxGDR/9T+BfM/PzmXkt8J2IeMkYXncXcFBEzIiIXYGTgRtHHfMl4DcjYpfqex4JPDiRxUs7a+G8mXR3TdqqrbtrEgvnzSyoIkmSJKm+RoXCZVSGdo7oBv7rhV6Umc8CHwaWUgl6izPz/og4KyLOqh7zIPBVYAVwJ/DPmXnfBNcv7ZT5s3v5+EmH0tvTTQC9Pd18/KRDGz/JjCRJkjROkTnxIysj4t7MPOyF2pqlv78/BwYGXvhASZIkSepAEXF3ZvbX29eonsKnIuLwmgJ+A3CGDUmSJElqMY2aaOZPgP+IiJFJYvYH3t2gc0mtacViF5GXJElSy2vU4vV3RcTBwEwqy0x8PzM3NeJcUktasRhuOgc2VTvIN66pbIPBUJIkSS2lIcNHI+JDwO6ZeV9mrgT2iIgPNuJcUktaduGvA+GITcOVdkmSJKmFNOqewvdn5oaRjcz8GfD+Bp1Laj0bB8fXLkmSJBWkUaHwRRERIxsRMQnYtUHnklrP5L7xtUuSJEkFaVQoXAosjoi5EfFm4AvALQ06l9R65p4PXd1bt3V1V9olSZKkFtKo2Uc/CpwJnE1lopnlVGYglcphZDIZZx+VJElSi2vU7KPPRcR3gFdSWYpib+CLjTiX1LJmLTAESpIkqeVNaCiMiFcDJwOnAOuBfwfIzN+ZyPNIkiRJkibGRPcUfh/4FnB8Zq4GiIg/neBzSJIkSZImyERPNPMu4H+B2yLisxExl8o9hZIkSZKkFjShPYWZeQNwQ0TsDswH/hTYLyL+EbghM782keeTXsiS5UMsWrqKtRuGmdrTzcJ5M5k/u7fosiRJkqSW0ZAlKTLzqcy8NjPfAfQB9wLnNuJc0rYsWT7EedevZGjDMAkMbRjmvOtXsmT5UNGlSZIkSS2jUesUbpGZT2TmP2Xmmxt9LqnWoqWrGN60eau24U2bWbR0VUEVSZIkSa2n4aFQKsraDcPjapckSZLKqFGL10uFm9rTzW/8/Ov8310WMzV+ytrcl08+u4C7X/qWokuTJEmSWoahUB3rM699iEPu/me64xkA+uKnfKLrn7nvtdMBRzNLkiRJ4PBRdbAjfvj3WwLhiO54hiN++PcFVSRJkiS1HkOhOtfGwfG1S5IkSSXUcqEwIo6NiFURsToinreMRUT8dkRsjIh7q3/OL6JOtYHJfeNrlyRJkkqopUJhREwCLgOOA14LnBIRr61z6Lcy87DqnwubWqTax9zzoat767au7kq7JEmSJKDFQiEwB1idmQ9n5jPAdcAJBdekdjVrARx/CUyeBkTl8fhLKu2SJEmSgNabfbQXWFOzPQgcWee4N0bE94C1wJ9n5v3NKE5taNYCQ6AkSZK0Ha0WCqNOW47avgd4RWb+IiLeBiwBDnreG0WcCZwJcMABB0x0nZIkSZLUEVpt+OggMK1mu49Kb+AWmfnzzPxF9fnNQFdE7Dv6jTLziszsz8z+KVOmNLJmSZIkSWpbrdZTeBdwUETMAIaAk4H31B4QES8HfpKZGRFzqATb9U2vVOOyZPkQi5auYu2GYab2dLNw3kzmz+4tuixJkiSp9FoqFGbmsxHxYWApMAm4KjPvj4izqvsvB34XODsingWGgZMzc/QQU7WQJcuHOO/6lQxv2gzA0IZhzrt+JYDBUJIkSSpYlCFP9ff358DAQNFllNZRF93K0Ibh57X39nRzx7lvLqAiSZIkqVwi4u7M7K+3r9XuKVQHWlsnEG6vXZIkSVLzGArVcFN7usfVLkmSJKl5DIVquIXzZtLdNWmrtu6uSSycN7OgiiRJkiSNaKmJZtSZRiaTcfZRSZIkqfUYCtUU82f3GgIlSZKkFuTwUUmSJEkqMUOhJEmSJJWYoVCSJEmSSsxQKEmSJEklZiiUJEmSpBIzFEqSJElSiRkKJUmSJKnEDIWSJEmSVGKGQkmSJEkqMUOhJEmSJJWYoVDNsWIxXHwIXNBTeVyxuOiKJEmSJAG7FF2AmmfJ8iEWLV3F2g3DTO3pZuG8mcyf3dv4E69YDDedA5uGK9sb11S2AWYtaPz5JUmSJG2TPYUlsWT5EOddv5KhDcMkMLRhmPOuX8mS5UONP/myC38dCEdsGq60S5IkSSqUobAkFi1dxfCmzVu1DW/azKKlqxp/8o2D42uXJEmS1DSGwpJYu2F4XO0TanLf+NolSZIkNU3LhcKIODYiVkXE6og4dzvHHRERmyPid5tZX7ua2tM9rvYJNfd86Bp1nq7uSrskSZKkQrVUKIyIScBlwHHAa4FTIuK12zjuE8DS5lbYvhbOm0l316St2rq7JrFw3szGn3zWAjj+Epg8DYjK4/GXOMmMJEmS1AJabfbROcDqzHwYICKuA04AHhh13B8DXwSOaG557WtkltFCZh+FSgA0BEqSJEktp9VCYS+wpmZ7EDiy9oCI6AVOBN6MoXBc5s/ubV4IlCRJktQWWmr4KBB12nLU9meAj2bm5jrH/vqNIs6MiIGIGFi3bt2EFdjWXEBekiRJ0iit1lM4CEyr2e4D1o46ph+4LiIA9gXeFhHPZuaS2oMy8wrgCoD+/v7RwbJ8XEBekiRJUh2t1lN4F3BQRMyIiF2Bk4Ebaw/IzBmZOT0zpwP/CXxwdCBUHS4gL0mSJKmOluopzMxnI+LDVGYVnQRclZn3R8RZ1f2XF1pgO3MBeUmSJEl1tFQoBMjMm4GbR7XVDYOZeXozauoIk/sqQ0brtUuSJEkqrVYbPqpGcQF5SZIkSXUYCsvCBeQlSZIk1dFyw0fVQC4gL0mSJGkUewolSZIkqcQMhZIkSZJUYoZCSZIkSSoxQ6EkSZIklZihUJIkSZJKzFAoSZIkSSVmKJQkSZKkEjMUSpIkSVKJGQolSZIkqcQMhZIkSZJUYoZCSZIkSSoxQ6EkSZIklZihUJIkSZJKzFAoSZIkSSVmKJQkSZKkEjMUSpIkSVKJGQolSZIkqcRaLhRGxLERsSoiVkfEuTwsg/oAACAASURBVHX2nxARKyLi3ogYiIiji6hTkiRJkjrBLkUXUCsiJgGXAW8BBoG7IuLGzHyg5rBlwI2ZmRExC1gMHNz8aiVJkiSp/bVaT+EcYHVmPpyZzwDXASfUHpCZv8jMrG7uDiSSJEmSpB3SaqGwF1hTsz1YbdtKRJwYEd8HvgK8t0m1SZIkSVLHabVQGHXantcTmJk3ZObBwHzgr+u+UcSZ1XsOB9atWzfBZUqSJElSZ2i1UDgITKvZ7gPWbuvgzPwm8KqI2LfOvisysz8z+6dMmTLxlUqSJElSB2i1UHgXcFBEzIiIXYGTgRtrD4iIAyMiqs8PB3YF1je9UkmSJEnqAC01+2hmPhsRHwaWApOAqzLz/og4q7r/cuBdwGkRsQkYBt5dM/GMJEmSJGkcogx5qr+/PwcGBoouQ5IkSZIKERF3Z2Z/vX0t1VMoSZKk8tm0aRODg4M8/fTTRZcitb3ddtuNvr4+urq6xvwaQ6EkSZIKNTg4yJ577sn06dOpTh0haQdkJuvXr2dwcJAZM2aM+XWtNtGMJEmSSubpp59mn332MRBKOyki2Geffcbd625PYQGWLB9i0dJVrN0wzNSebhbOm8n82b1FlyVJklQYA6E0MXbke8mewiZbsnyI865fydCGYRIY2jDMedevZMnyoaJLkyRJKq33vve9vOxlL+OQQw7Z5jGZyTnnnMOBBx7IrFmzuOeee7bs++pXv8rMmTM58MADueiii8Z0zj322GOn695RV199NWvXbnM58LY0lq/Btddey6xZs5g1axZvetOb+N73vrdl3/Tp0zn00EM57LDD6O+vOx/L83TK19BQ2GSLlq5ieNPmrdqGN21m0dJVBVUkSZKk008/na9+9avbPeaWW27hoYce4qGHHuKKK67g7LPPBmDz5s186EMf4pZbbuGBBx7gC1/4Ag888MAO1bF58+YXPmgCdFooHOvXYMaMGXzjG99gxYoV/OVf/iVnnnnmVvtvu+027r33XnZm5YJ2/BoaCpts7YbhcbVLkiRpa0uWD3HURbcy49yvcNRFt07IiKtjjjmGvffee7vHfOlLX+K0004jInjDG97Ahg0beOyxx7jzzjs58MADeeUrX8muu+7KySefzJe+9KXnvf5HP/oRb3zjGzniiCP4y7/8yy3t//3f/83v/M7v8J73vIdDDz2Up59+mjPOOINDDz2U2bNnc9tttwGVEHDCCSdw7LHHMnPmTP7qr/5qy3v87d/+LYcccgiHHHIIn/nMZwB45JFHtur5/NSnPsUFF1zAf/7nfzIwMMDv//7vc9hhhzE8XMDvoSsWw8WHwAU9lccVi3fq7cb6NXjTm97EXnvtBcAb3vAGBgcHx3WeTv0aek9hk03t6WaoTgCc2tNdQDWSJEntZeRWnJGRVyO34gANn6NhaGiIadOmbdnu6+tjaGiobvt3v/vd573+Ix/5CGeffTannXYal1122Vb77rzzTu677z5mzJjBpz/9aQBWrlzJ97//fd761rfygx/8YKvjXvKSl3DEEUfw9re/nYjgc5/7HN/97nfJTI488kh+67d+a0v4Ge13f/d3ufTSS/nUpz415mGSE2rFYrjpHNhU/Z1445rKNsCsBTv0lmP9GtS68sorOe6447ZsRwRvfetbiQg+8IEPPK8XETr3a2hPYZMtnDeT7q5JW7V1d01i4byZBVUkSZLUPoq8FSczn9cWEdtsH+2OO+7glFNOAeDUU0/dat+cOXO2LCFw++23b9l/8MEH84pXvGJLoHjLW97CPvvsQ3d3NyeddBK33347t99+OyeeeCK77747e+yxByeddBLf+ta3du7DNtKyC38dCEdsGq6076Cxfg1G3HbbbVx55ZV84hOf2NJ2xx13cM8993DLLbdw2WWX8c1vfvN5r+vUr6E9hU02f3YvvWu+zLR7FvGyXMfjMYU1hy/kiNnHFl2aJElSyyvyVpy+vj7WrFmzZXtwcJCpU6fyzDPP1G2vZ1tBZffdd9/yvF7A2dbrtxVKAXbZZReee+65LdvjXaagYTZuY8jmttrHYFtfm3pWrFjB+973Pm655Rb22WefLe0jx7/sZS/jxBNP5M477+SYY4553us78WtoT2GzrVjMESs/xstZx4sCXs46jlj5sZ0eRy1JklQG27rlphm34rzzne/kmmuuITP5zne+w+TJk9l///054ogjeOihh/jRj37EM888w3XXXcc73/nO573+qKOO4rrrrgMqs2BuyzHHHLNl/w9+8AMeffRRZs6sjCr7+te/zhNPPMHw8DBLlizhqKOO4phjjmHJkiX88pe/5KmnnuKGG27gN3/zN9lvv/14/PHHWb9+Pb/61a/48pe/vOUce+65J08++eRE/vWM3eS+8bWPwVi/Bo8++ignnXQS//Iv/8KrX/3qLe1PPfXUlr+Pp556iq997Wt1Z6Lt1K+hobDZGtBdLkmSVBaNuhXnlFNO4Y1vfCOrVq2ir6+PK6+8EoDLL7+cyy+/HIC3ve1tvPKVr+TAAw/k/e9/P//wD/8AVHpzLr30UubNm8drXvMaFixYwOte97rnnePv/u7vuOyyyzjiiCPYuHHjNmv54Ac/yObNmzn00EN597vfzdVXX82LX/xiAI4++mhOPfVUDjvsMN71rnfR39/P4Ycfzumnn86cOXM48sgjed/73sfs2bPp6uri/PPP58gjj+Qd73gHBx988JZznH766Zx11lnFTDQz93zoGhXiu7or7Ttoe1+D2q/hhRdeyPr16/ngBz+41dITP/nJTzj66KN5/etfz5w5c3j729/Oscc+fyRfp34NY3tdm52iv78/d2Za2Ql1QQ9Q7+884IINza5GkiSpcA8++CCvec1rxnz8kuVDLFq6irUbhpna083CeTMbPslMK7j66qsZGBjg0ksvLbqUnbdicaVTZONgpYdw7vk7PMlMO2nW17De91RE3J2ZdWel8Z7CZpvcV5lhqV67JEmSXtD82b2lCIEdbdaCUoTAdmFPYbONnoIXKt3lx1/iN4YkSSql8fYUStq+8fYUek9hs81aUAmAk6cBUXk0EEqSJEkqiMNHi2B3uSRJ0lYyc7vrykkamx0ZCWpPoSRJkgq12267sX79+h36ZVbSr2Um69evZ7fddhvX6+wplCRJUqH6+voYHBxk3bp1RZcitb3ddtuNvr7xTWJpKJQkSVKhurq6mDFjRtFlSKXl8FFJkiRJKjFDoSRJkiSVmKFQkiRJkkqsFIvXR8Q64MdF11HHvsBPiy5CHcvrS43k9aVG8xpTI3l9qZFa9fp6RWZOqbejFKGwVUXEQGb2F12HOpPXlxrJ60uN5jWmRvL6UiO14/Xl8FFJkiRJKjFDoSRJkiSVmKGwWFcUXYA6mteXGsnrS43mNaZG8vpSI7Xd9eU9hZIkSZJUYvYUSpIkSVKJGQoLEBHHRsSqiFgdEecWXY86T0Q8EhErI+LeiBgouh61t4i4KiIej4j7atr2joivR8RD1ce9iqxR7Wsb19cFETFU/Rl2b0S8rcga1b4iYlpE3BYRD0bE/RHxkWq7P8O007ZzfbXdzzCHjzZZREwCfgC8BRgE7gJOycwHCi1MHSUiHgH6M7MV18hRm4mIY4BfANdk5iHVtk8CT2TmRdX/3NorMz9aZJ1qT9u4vi4AfpGZnyqyNrW/iNgf2D8z74mIPYG7gfnA6fgzTDtpO9fXAtrsZ5g9hc03B1idmQ9n5jPAdcAJBdckSduUmd8EnhjVfALw+erzz1P5R1Aat21cX9KEyMzHMvOe6vMngQeBXvwZpgmwneur7RgKm68XWFOzPUibXjxqaQl8LSLujogziy5GHWm/zHwMKv8oAi8ruB51ng9HxIrq8FKH9mmnRcR0YDbwXfwZpgk26vqCNvsZZihsvqjT5hheTbSjMvNw4DjgQ9XhWZLULv4ReBVwGPAY8Oliy1G7i4g9gC8Cf5KZPy+6HnWWOtdX2/0MMxQ23yAwrWa7D1hbUC3qUJm5tvr4OHADlWHL0kT6SfVeipF7Kh4vuB51kMz8SWZuzszngM/izzDthIjoovIL+7WZeX212Z9hmhD1rq92/BlmKGy+u4CDImJGROwKnAzcWHBN6iARsXv1ZmciYnfgrcB923+VNG43An9Yff6HwJcKrEUdZuSX9aoT8WeYdlBEBHAl8GBm/m3NLn+Gaadt6/pqx59hzj5agOq0tJ8BJgFXZebfFFySOkhEvJJK7yDALsC/eY1pZ0TEF4DfBvYFfgJ8DFgCLAYOAB4Ffi8znSxE47aN6+u3qQy7SuAR4AMj939J4xERRwPfAlYCz1Wb/4LKfV/+DNNO2c71dQpt9jPMUChJkiRJJebwUUmSJEkqMUOhJEmSJJWYoVCSJEmSSsxQKEmSJEklZiiUJEmSpBIzFEqSNEYRsTki7q35c+4Evvf0iGj5tawkSZ1nl6ILkCSpjQxn5mFFFyFJ0kSyp1CSpJ0UEY9ExCci4s7qnwOr7a+IiGURsaL6eEC1fb+IuCEivlf986bqW02KiM9GxP0R8bWI6C7sQ0mSSsNQKEnS2HWPGj767pp9P8/MOcClwGeqbZcC12TmLOBa4JJq+yXANzLz9cDhwP3V9oOAyzLzdcAG4F0N/jySJBGZWXQNkiS1hYj4RWbuUaf9EeDNmflwRHQB/5uZ+0TET4H9M3NTtf2xzNw3ItYBfZn5q5r3mA58PTMPqm5/FOjKzP+38Z9MklRm9hRKkjQxchvPt3VMPb+qeb4Z7/2XJDWBoVCSpInx7prHb1ef/w9wcvX57wO3V58vA84GiIhJEfHSZhUpSdJo/g+kJElj1x0R99ZsfzUzR5aleHFEfJfKf7ieUm07B7gqIhYC64Azqu0fAa6IiD+i0iN4NvBYw6uXJKkO7ymUJGknVe8p7M/MnxZdiyRJ4+XwUUmSJEkqMXsKJUmSJKnE7CmUJKkqInJk4XlJksrCUChJ6hgRsTQiLqzTfkJE/G9EOMGaJEmjGAolSZ3kauDUiIhR7acC12bms80vaWJExKSia5AkdSZDoSSpkywB9gZ+c6QhIvYC3gFcExFzIuLbEbEhIh6LiEsjYtexvHFEnBERD0bEkxHxcER8YNT+EyLi3oj4eUT8MCKOrbbvHRGfi4i1EfGziFhSbT89Im4f9R5bhq9GxNUR8Y8RcXNEPAX8TkS8PSKWV8+xJiIuGPX6oyPif6qfb031HEdExE9qe0kj4l2jltaQJJWYoVCS1DEycxhYDJxW07wA+H5mfo/KmoB/CuwLvBGYC3xwjG//OJVw+VIq6w1eHBGHA0TEHOAaYCHQAxwDPFJ93b8ALwFeB7wMuHgcH+k9wN8Ae1JZ+P6p6mfrAd4OnB0R86s1HADcAvw9MAU4DLg3M+8C1gNvqXnfP6jWJUmSs49KkjpLRBwNfAV4eWYOR8QdwH9m5vPCWET8CfBbmXlidTuBgzJz9RjOswS4LTP/LiL+CfhlZv7pqGP2B4aAfTLzZ6P2nQ68LzOPrmnbcv6IuBp4UWbWBtzRNXwGyMz804g4D5gz8llGHfdRYFZm/n5E7A0MAq/KzMde6HNKkjqfPYWSpI6SmbcD64ATIuKVwBHAvwFExKsj4svVSWd+Dvx/VHoNX1BEHBcR34mIJyJiA/C2mtdOA35Y52XTgCdGB8JxWDOqhiMj4raIWBcRG4GzxlADwL8Cx0fEHlR6Tr9lIJQkjTAUSpI60TVUhlmeCnwtM39Sbf9H4PtUeuNeCvwFMHpSmueJiBcDXwQ+BeyXmT3AzTWvXQO8qs5L1wB7R0RPnX1PURlWOnKOl9c5ZvRwnn8DbgSmZeZk4PIx1EBmDgHfBk6k8nfi0FFJ0haGQklSJ7oG+D/A+4HP17TvCfwc+EVEHAycPcb32xV4MZUeyGcj4jjgrTX7rwTOiIi5EfGiiOiNiIOrvXG3AP8QEXtFRFdEHFN9zfeA10XEYRGxG3DBGOrYk0rP49PV+xjfU7PvWuD/RMSCiNglIvaJiMNq9l8D/F/gUOCGMX5uSVIJGAolSR0nMx8B/gfYnUrP2og/pxKkngQ+C/z7GN/vSeAcKpPY/Kz6HjfW7L+T6uQzwEbgG8ArqrtPBTZR6aF8HPiT6mt+AFwI/BfwEJWJZF7IB4ELI+JJ4PxqPSM1PEplSOufAU8A9wKvr3ntDdWabsjMp8byuSVJ5eBEM5IklURE/BD4QGb+V9G1SJJahz2FkiSVQES8i8o9ircWXYskqbXs8sKHSJKkdhYR/w28Fjg1M58ruBxJUotx+KgkSZIklZjDRyVJkiSpxAyFkiRJklRipbincN99983p06cXXYYkSZIkFeLuu+/+aWZOqbevFKFw+vTpDAwMFF2GJEmSJBUiIn68rX0OH5UkSZKkEjMUSpIkSVKJNTQURsSxEbEqIlZHxLl19p8QESsi4t6IGIiIo2v2PRIRK0f21bTvHRFfj4iHqo97NfIzSJIkSVIna1gojIhJwGXAcVQWzD0lIl476rBlwOsz8zDgvcA/j9r/O5l5WGb217SdCyzLzIOqr39e2JQkSZIkjU0jewrnAKsz8+HMfAa4Djih9oDM/EVmZnVzdyB5YScAn68+/zwwf4LqlSRJkqTSaWQo7AXW1GwPVtu2EhEnRsT3ga9Q6S0ckcDXIuLuiDizpn2/zHwMoPr4sgmvXJIkSZJKopGhMOq0Pa8nMDNvyMyDqfT4/XXNrqMy83Aqw08/FBHHjOvkEWdW71McWLdu3XheKkmSpEZZsRguPgQu6Kk8rlhcdEVS6TUyFA4C02q2+4C12zo4M78JvCoi9q1ur60+Pg7cQGU4KsBPImJ/gOrj49t4vysysz8z+6dMqbtGoyRJkpppxWK46RzYuAbIyuNN5xgMpYI1MhTeBRwUETMiYlfgZODG2gMi4sCIiOrzw4FdgfURsXtE7Flt3x14K3Bf9WU3An9Yff6HwJca+BkkSZI0UZZdCJuGt27bNFxpl1SYXRr1xpn5bER8GFgKTAKuysz7I+Ks6v7LgXcBp0XEJmAYeHdmZkTsB9xQzYu7AP+WmV+tvvVFwOKI+CPgUeD3GvUZJEmSNIE2Do6vXVJTNCwUAmTmzcDNo9our3n+CeATdV73MPD6bbznemDuxFYqSZKkhpvcVx06WqddUmEauni9JEmStMXc86Gre+u2ru5Ku6TCGAolSZLUHLMWwPGXwORpQFQej7+k0i6pMA0dPipJkiRtZdYCQ6DUYuwplCRJkqQSMxRKkiRJUokZCiVJkiSpxAyFkiRJklRihkJJkiRJKjFDoSRJkiSVmKFQkiRJkkrMUChJkiRJJWYolCRJkqQSMxRKkqStrVgMFx8CF/RUHlcsLroiSVID7VJ0AZIkaRtWLIZlF8LGQZjcB3PPh1kLGn/Om86BTcOV7Y1rKtvQ+HNLkgphT6EkSa1oJJxtXAPkr8NZo3vtll3460A4YtNwpV2S1JEMhZIktaKiwtnGwfG1S5LanqFQkqRWVFQ4m9w3vna1rzLeO1rGzyyNgaFQkqRWVFQ4m3s+dHVv3dbVXWlX5yhqeHKRyviZpTEyFEqS1IqKCmezFsDxl8DkaUBUHo+/xElmOk0Z7x0t42eWxsjZRyVJakUjIazZs4+OnNsQ2NnKeO9oGT+zNEaGQkmSWpXhTI0yua86jLJOe6cq42eWxsjho5IkSWVTxntHy/iZpTEyFEqSJJVNGe8dLeNnlsYoMrPoGhquv78/BwYGii5DkiRJkgoREXdnZn+9fQ3tKYyIYyNiVUSsjohz6+w/ISJWRMS9ETEQEUdX26dFxG0R8WBE3B8RH6l5zQURMVR9zb0R8bZGfgZJkiRJ6mQNm2gmIiYBlwFvAQaBuyLixsx8oOawZcCNmZkRMQtYDBwMPAv8WWbeExF7AndHxNdrXntxZn6qUbVLkiRJUlk0sqdwDrA6Mx/OzGeA64ATag/IzF/kr8ev7g5ktf2xzLyn+vxJ4EGgt4G1SpIkSVIpNTIU9gK18/4OUifYRcSJEfF94CvAe+vsnw7MBr5b0/zh6rDTqyJir4ksWpIkSZLKpJGhMOq0PW9Wm8y8ITMPBuYDf73VG0TsAXwR+JPM/Hm1+R+BVwGHAY8Bn6578ogzq/cpDqxbt27HP4UkSWqeFYvh4kPggp7K44rFRVfUUEuWD3HURbcy49yvcNRFt7Jk+VDRJUkqoUaGwkFgWs12H7B2Wwdn5jeBV0XEvgAR0UUlEF6bmdfXHPeTzNycmc8Bn6UyTLXe+12Rmf2Z2T9lypSd/zSSJKmxViyGm86pLjCelcebzunYYLhk+RDnXb+SoQ3DJDC0YZjzrl9pMOxEJfvPDrWfRobCu4CDImJGROwKnAzcWHtARBwYEVF9fjiwK7C+2nYl8GBm/u2o1+xfs3kicF8DP4MkSWqWZRfCpuGt2zYNV9o70KKlqxjetHmrtuFNm1m0dFVBFakhSvafHWpPDZt9NDOfjYgPA0uBScBVmXl/RJxV3X858C7gtIjYBAwD767ORHo0cCqwMiLurb7lX2TmzcAnI+IwKkNRHwE+0KjPIEmSmmjj4Pja29zaDcPjaleb2t5/dsxaUExNaowViytf142DMLkP5p7fNl/jhoVCgGqIu3lU2+U1zz8BfKLO626n/j2JZOapE1ymJEnb1sb/yLedyX3V3pQ67R1oak83Q3UC4NSe7gKqUcOU7D87SmukR3jkPwBGeoShLf7NaOji9ZIktTWHfTXX3POha1Qg6uqutHeghfNm/v/t3XuUHHWZ8PHvQ0jMcDHBGBAyQaJgUJNIdEAlLoosN10JREXARREUgnq8vO/Ja3yPssjue4iC4gUUoyDqumJWIcQLIAfxAghkEjDc5KKwZBIkIRpUCObC8/7RPWEyzCQ9ydRUuuv7OWdOd/2qquup6Zqefup3o234sE3K2oYPY9aRE0uKSIXo76ZGi97s6Fa5QZSavPm7SaEkSf1p8n/yTWfK8fC2L8Oo8UDUHt/25aa4y741jp06jnNnTGbc6DYCGDe6jXNnTObYqU7N3FIqdrMDKjqIUpPXCBfafFSSpKbW5P/km9KU41s2CezLsVPHmQS2uu7ruULN0Dc3iFLLXu9N3vzdpFCSpP40+T95SduJit3sqOQgSoedtWmfQmiqGmGbj0qS1J8KNvtSdVSuz5eGTH+DJbX0IEpN3vzdmkJJkvpTwWZfqobuPl/dTfy6+3wBrdu8T0Nm1pETN7m+oCKDKDVxjbBJoSSpcVWcnqGJ/8lL/alkny8Nme5r6Lxr72P56jXsNbqNWUdO9NrajpkUSpIa0+RzMKlx829f5pe5FlfJPl8aUg6i1FxMCiVJjdnc9AwmhS3DZoXVsNfoNpb1kQC2dJ8vvOEh9ceBZiRJjXF6hkrYXLNCtY5ZR06kbfiwTcpavc9XJefOkxpkUihJakx/0zA4PUNLsVnhEFsyDy6YBGePrj0umTckhz126jjOnTGZcaPbCGDc6DbOnTG5pWvNqnrDY+GCr/Ons/flmX8bxZ/O3peFC75edkjaDtl8VJLUmCafg0mNqWqzwlKU3E+3an2+qnjDY+GCrzNp0adoi7UQ8CJWMmrRp1gIHHjMGWWHp+2INYWStLVKusNfmiafg0mNqWKzwtJsrp+uBl0V584bv/i8WkLYQ1usZfzi80qKSNsrawolaWtUdSROp2doeQ4lP4Tspzukqjh33u65EqKv8seHPhht10wKJWlrOBKnWljVmhVCSaNSjmqv3VDqq1yDroo3PFbEWF7Eyj7KX8iLSohH2y+TQknaGt7hH3pL5tWS7ie6al+aDzvLBFyDorRpOOynO+SqdsNj6atnMaq7T2HdmhzB0tfMMiksQDNPeWKfQknNr4y+fWWPxFm1/ozdzXWfWArks811W/28NSRKG5XSfroq2IHHnMFdr/kP/sRYnsngT4zlrtf8R2sPMlPS/8dmn/IkMrPsGArX0dGRnZ2dZYchqQi9+/ZB7U570V+syjpu2ccuywWT+mlmNx4+ftfQx6OWMmH2T+nr21AAD81561CHI2lrlfj/cdqcX/Q5cvO40W3cNPvNhR67URGxKDM7+lpnTaGk5lbW6H1l3uGv4oiFNtdVgao4KqWG1vzblzFtzi+YMPunTJvzi6apPdomZdTYlfj/sdmnPLFPoaTmVmayUNZInFVMkCo6IEcz909pJlUclVJDp7Q+q2UqaYTufKKrr8FW+y0fTM0+x6s1hZKaW9l9+8pQxXM+7KxaE6CeWnxAjmbvn7K1yqhROXbqOM6dMZlxo9sIas29zp0xuXW/sGtIldZntUwl1dg9xgsHVD6Ymn2OV5NCSc2tgslCJc+5ggNyVPGLZJmJ8LFTx3HT7Dfz0Jy3ctPsN5sQatA0e7PCrVJSi5Zz176Tp3LEJmVP5QjOXfvOQo8LzX9zyeajkppbd1JQpakKqnjOUF5z3ZJU8Yvk5hLhwr9YOeXJkKlas+hmb1a4VUpq8t/5/MOZ/Vf4PzvOY69YxfIcw+fWH8+i5x9e6HG7NfOUJyaFkppfxZIFoJrnXDFV/CJZWiJcUv+nKqpi/7pK9lktaQ7O2u96LQvWvmFjWdvwYZzbyr/rQWLzUUmDp2pz50kFavb+KVujtFFAqziib0mq2Cy62ZsVbpWSmvxX8nc9SAqtKYyIo4AvAcOAb2bmnF7rpwP/DjwDrAc+lpk3bm7fiHgB8ANgH+Bh4PjM/EuR5yGpAd5plwZV95eYKjWzK61GpYoj+pakis2iobmbFW61klq0VPJ3PQgKSwojYhhwEXA40AUsjIgFmXlPj82uBxZkZkbEFGAesP8W9p0NXJ+ZcyJidn35E0Wdh6QGbe5Ou0mhmliZ/Z+q9uWmtES4olOelKGKzaKlZlBkTeFBwIOZ+UeAiLgcmA5sTAoz8+89tt8ZyAb2nQ68qb7dt4FfYlIobbRwwdcZv/g8ds+V/QggsAAAH7ZJREFUrIixLH31LA485oziD+yddrWgKvZ/KlspiXBJ/Z+qqJL966QmUGSfwnFAz9tuXfWyTUTEcRHxe+CnwKkN7LtHZj4KUH/cfZDjlprWwgVfZ9KiT/EiVrJDwItYyaRFn2Lhgq8Xf/Aqzp2nllfF/k+VVMEpT8piny9p+1RkTWH0UZbPKci8ErgyIg6h1r/wnxvdd7MHjzgdOB1g7733HsiuUtMav/g82mLtJmVtsZbxi8+DomsLvdOuFlTV/k+V5Ii+Q6ZqzaKlZlBkTWEXML7HcjuwvL+NM/PXwEsj4oVb2PexiNgToP64op/Xm5uZHZnZMXbs2K0/C6mJ7J4r+yl/vPiDe6ddLai00TAlSRpCRSaFC4H9ImJCRIwATgAW9NwgIvaNiKg/fzUwAli1hX0XAO+tP38vcFWB5yBtvRKmZ1gRfd8AWREvLPzYQC0B/PhdcPbq2qMJoZpcFaeFkCRVT2FJYWauBz4MXAvcC8zLzLsjYmZEzKxv9nbgroi4g9poo+/Kmj73re8zBzg8Ih6gNjrpJtNcSNuF7ukZnlgK5LPTMxScGC599SzW5IhNytbkCJa+elahx5Valf2fJElVEJkD6qrXlDo6OrKzs7PsMFQlF0zqZ3jz8bUatAI9O/ro46yIFw7d6KOSJEnabkXEoszs6GtdoZPXS5VV4vQMBx5zxsZBZV5U/5EkSZL6U2SfQqm6nJ5BkiRJTcKkUCrCYWfVpmPoyekZJEmStB0yKZSK4PQMkiRJahL2KZSK4kTIkiRJagLWFKq1lTBXoCRJktRMrClU6+qeK3Ddmtpy91yBYA2eJEmSVGdNoVrX9ec8mxB2W7emVi5JkiQJMClUKytxrkBJkiSpWZgUqnU5V6AkSZK0RSaFal3OFShJkiRtkUmhWpdzBUqSJElb5Oijam3OFShJkiRtljWFkiRJklRhJoWSJEmSVGEmhZIkSZJUYfYplDRo5t++jPOuvY/lq9ew1+g2Zh05kWOnjis7LLUIry9JkophUihpUMy/fRmfvOJO1qzbAMCy1Wv45BV3AvjFXdvM60uSpOLYfFTSoDjv2vs2fmHvtmbdBs679r6SIlIr8fqSJKk4W0wKI+JfIsLkUdJmLV+9ZkDl0kB4fUmSVJxGkr0TgAci4nMR8fKiA5LUnPYa3TagcmkgvL4kSSrOFpPCzPxXYCrwB+BbEfHbiDg9InYtPDpJTWPWkRNpGz5sk7K24cOYdeTEkiJSK/H6kiSpOA0NNJOZf42IHwFtwMeA44BZEfHlzPxKkQFK28LRCodO9+/V37eK4PUlSVJxIjM3v0HE24BTgZcC3wW+nZkrImIn4N7MfHHxYW6bjo6O7OzsLDsMDbHeoxVCrWbh3BmT/SIpSZKkSomIRZnZ0de6RvoUvhO4IDOnZOZ5mbkCIDOfopYsbu7AR0XEfRHxYETM7mP9uyNiSf3n5oh4Vb18YkTc0ePnrxHxsfq6syNiWY91b2ngHFSy+bcvY9qcXzBh9k+ZNucXzL99WeHHdLRCSZIkacsaaT76b8Cj3QsR0QbskZkPZ+b1/e0UEcOAi4DDgS5gYUQsyMx7emz2EPDGzPxLRBwNzAVem5n3AQf0eJ1lwJU99rsgM89v6AxVurLmF3O0QrUym0ZLkqTB0khN4X8Dz/RY3lAv25KDgAcz84+ZuRa4HJjec4PMvDkz/1JfvAVo7+N1DgP+kJn/08AxtR0qq8bO0QrVqrpvtCxbvYbk2RstQ1EDL0mSWk8jSeGO9aQOgPrzEQ3sNw5Y2mO5q17Wn9OAq/soPwH4fq+yD9ebnF4aEbs1EItKVFaNnaMVqlXZNFqSJA2mRpLClRFxTPdCREwHHm9gv+ijrM9RbSLiUGpJ4Sd6lY8AjmHTmsmvURv05gBqzVo/389rnh4RnRHRuXLlygbCVVHKqrE7duo4zp0xmXGj2whg3Og2B5lRS7BptCRJGkyN9CmcCXwvIi6klugtBd7TwH5dwPgey+3A8t4bRcQU4JvA0Zm5qtfqo4HFmflYd0HP5xHxDeAnfR08M+dS66NIR0fH5odYVaFmHTmxz1FAh6LG7tip40wC1XL2Gt3Gsj4SQJtGS5KkrbHFpDAz/wC8LiJ2oTaFxd8afO2FwH4RMYHaQDEnACf13CAi9gauAE7OzPv7eI0T6dV0NCL2zMzugW+OA+5qMB6VpKrzizkQiIpS5o0WSZLUehqavD4i3gq8EhgZUWsVmpnnbG6fzFwfER8GrgWGAZdm5t0RMbO+/mLgLGAM8NX6667vnjujPg/i4cAZvV76cxFxALWmqA/3sV7boarV2JU14qqqoao3WiRJUjEambz+YmAn4FBqzTzfAdyWmacVH97gcPJ6DbVpc37RZ/O+caPbuGn2m0uISJIkSVW2ucnrG6kpPDgzp0TEksz8TER8nlqTT0n9cCCQarCJsCRJagWNjD76dP3xqYjYC1gHTCguJKn5OUdi63OuQEmS1CoaSQp/HBGjgfOAxdT68fWeN1BSD86R2PqcK1CSJLWKzTYfjYgdgOszczXwo4j4CTAyM58YkuikJuVAIK3PJsKSJKlVbDYpzMxn6n0IX19f/gfwj6EITGp2VRtxtWqcK1CSJLWKRpqP/jwi3h7dc1FIkmwiLEmSWkYjo4/+L2BnYH1EPA0EkJn5/EIjk6TtmE2EJUlSq9hiUpiZuw5FIJLUbGwiLEmSWsEWk8KIOKSv8sz89eCHo5a1ZB5cfw480QWj2uGws2DK8WVHJUmSJFVeI81HZ/V4PhI4CFgEvLmQiNR6lsyDH38E1tUH5XhiaW0ZTAwlSZKkkjXSfPRtPZcjYjzwucIiUuu5/pxnE8Ju69bUyk0KJUmSpFI1UlPYWxcwabADUQt7omtg5ZK2W/NvX+bgOpIktZhG+hR+Bcj64g7AAcDvigxKLWZUe63JaF/lkprG/NuX8ckr7mTNug0ALFu9hk9ecSeAiaEkSU2skXkKO6n1IVwE/Bb4RGb+a6FRqbUcdhYM7zWh9/C2WrmkpnHetfdtTAi7rVm3gfOuva+kiCRJ0mBopPnoD4GnM3MDQEQMi4idMvOpYkNTy+juN+joo1JTW756zYDKJUlSc2gkKbwe+Gfg7/XlNuDnwMFFBaVilNoXaMrxJoFSk9trdBvL+kgA9xrd1sfWkiSpWTTSfHRkZnYnhNSf71RcSCrC/NuXceOVX+UHT32APzzvJH7w1Ae48cqvMv/2ZWWHJqlJzDpyIm3Dh21S1jZ8GLOOnFhSRJIkaTA0khQ+GRGv7l6IiNcAthVqMnf8dC7nxFzad3icHQLad3icc2Iud/x0btmhSWoSx04dx7kzJjNudBsBjBvdxrkzJjvIjCRJTa6R5qMfA/47IpbXl/cE3lVcSCrC+9f+JzvtsHaTsp1iLe9f+5/AZ8oJSlLTOXbqOJNASZJaTCOT1y+MiP2BiUAAv8/MdYVHpkG11w6rBlQuNRPnzpMkSdp6W2w+GhEfAnbOzLsy805gl4j4YPGhaTA93faiAZVLzaJ77rxlq9eQPDt3nv1lJUmSGtNIn8IPZObq7oXM/AvwgeJCUhF2Ovoc1g8buUnZ+mEj2enoc0qKSBoczp0nSZK0bRpJCneIiOheiIhhwIjiQlIhphzPjtO/AqPGAwGjxteWnSZCTc658yRJkrZNIwPNXAvMi4iLgQRmAlcXGpWK4VyBakHOnSdJkrRtGqkp/AS1CezPBD4ELKE2gf0WRcRREXFfRDwYEbP7WP/uiFhS/7k5Il7VY93DEXFnRNwREZ09yl8QEddFxAP1x90aiUVSa3LuPEmSpG2zxaQwM58BbgH+CHQAhwH3bmm/ejPTi4CjgVcAJ0bEK3pt9hDwxsycAvw70HvSvEMz84DM7OhRNhu4PjP3o5asPifZlFQdzp0nSZK0bfptPhoRLwNOAE4EVgE/AMjMQxt87YOABzPzj/XXuxyYDtzTvUFm3txj+1uA9gZedzrwpvrzbwO/pFabKaminDtPkiRp622upvD31GoF35aZb8jMrwAbNrN9b+OApT2Wu+pl/TmNTfsqJvDziFgUEaf3KN8jMx8FqD/uPoCYJEmSJEk9bG6gmbdTqym8ISKuAS6nNnl9o/raNvvcMOJQaknhG3oUT8vM5RGxO3BdRPw+M3/d8MFrieTpAHvvvXfjUUuSJElShfRbU5iZV2bmu4D9qTXR/DiwR0R8LSKOaOC1u4DxPZbbgeW9N4qIKcA3gemZuarH8ZfXH1cAV1JrjgrwWETsWd93T2BFP/HPzcyOzOwYO3ZsA+FKkiRJUvU0MtDMk5n5vcz8F2qJ3R00NrjLQmC/iJgQESOo1Tou6LlBROwNXAGcnJn39yjfOSJ27X4OHAHcVV+9AHhv/fl7gasaiEWSJEmS1IdG5incKDP/DHy9/rOlbddHxIepzXM4DLg0M++OiJn19RcDZwFjgK9GBMD6+kijewBX1st2BP4rM6+pv/QcavMmngY8ArxzIOcgSZIkSXpWZPbZza+ldHR0ZGdn55Y3lCRJkqQWFBGLek31t1Ejk9dLkiRJklqUSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVWKFJYUQcFRH3RcSDETG7j/Xvjogl9Z+bI+JV9fLxEXFDRNwbEXdHxEd77HN2RCyLiDvqP28p8hwkSZIkqZXtWNQLR8Qw4CLgcKALWBgRCzLznh6bPQS8MTP/EhFHA3OB1wLrgf+dmYsjYldgUURc12PfCzLz/KJilyRJkqSqKLKm8CDgwcz8Y2auBS4HpvfcIDNvzsy/1BdvAdrr5Y9m5uL6878B9wLjCoxVkiRJkiqpyKRwHLC0x3IXm0/sTgOu7l0YEfsAU4FbexR/uN7k9NKI2G3bQ5UkSZKkaioyKYw+yrLPDSMOpZYUfqJX+S7Aj4CPZeZf68VfA14KHAA8Cny+n9c8PSI6I6Jz5cqVW3cGkiRJktTiikwKu4DxPZbbgeW9N4qIKcA3gemZuapH+XBqCeH3MvOK7vLMfCwzN2TmM8A3qDVTfY7MnJuZHZnZMXbs2EE5IUmSJElqNUUmhQuB/SJiQkSMAE4AFvTcICL2Bq4ATs7M+3uUB3AJcG9mfqHXPnv2WDwOuKug+CVJkiSp5RU2+mhmro+IDwPXAsOASzPz7oiYWV9/MXAWMAb4ai0PZH1mdgDTgJOBOyPijvpL/t/M/BnwuYg4gFpT1IeBM4o6B0mSJElqdZHZZze/ltLR0ZGdnZ1lhyFJkiRJpYiIRfUKuOcodPJ6SZIkSdL2zaRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqrNCkMCKOioj7IuLBiJjdx/p3R8SS+s/NEfGqLe0bES+IiOsi4oH6425FnoMkSZIktbLCksKIGAZcBBwNvAI4MSJe0Wuzh4A3ZuYU4N+BuQ3sOxu4PjP3A66vL0uSJEmStkKRNYUHAQ9m5h8zcy1wOTC95waZeXNm/qW+eAvQ3sC+04Fv159/Gzi2wHOQJEmSpJZWZFI4DljaY7mrXtaf04CrG9h3j8x8FKD+uPugRCtJkiRJFbRjga8dfZRlnxtGHEotKXzDQPft9+ARpwOnA+y9994D2VWSJEmSKqPImsIuYHyP5XZgee+NImIK8E1gemauamDfxyJiz/q+ewIr+jp4Zs7NzI7M7Bg7duw2nYgkSZIktaoik8KFwH4RMSEiRgAnAAt6bhARewNXACdn5v0N7rsAeG/9+XuBqwo8h2IsmQcXTIKzR9cel8wrOyJJkiRJFVVY89HMXB8RHwauBYYBl2bm3RExs77+YuAsYAzw1YgAWF+v3etz3/pLzwHmRcRpwCPAO4s6h0IsmQc//gisW1NbfmJpbRlgyvHlxSVJkiSpkiJzQF31mlJHR0d2dnaWHUbNBZNqiWBvo8bDx+8a+ngkSZIktbyIWJSZHX2tK3TyevXhia6BlUuSJElSgUwKh9qo9oGVS5IkSVKBTAqH2mFnwfC2TcuGt9XKJUmSJGmImRQOtSnHw9u+XOtDSNQe3/ZlB5mRJEmSVIoiJ69Xf6YcbxIoSZJUt27dOrq6unj66afLDkVqeiNHjqS9vZ3hw4c3vI9JoSRJkkrV1dXFrrvuyj777EN9mjJJWyEzWbVqFV1dXUyYMKHh/Ww+KkmSpFI9/fTTjBkzxoRQ2kYRwZgxYwZc625SKEmSpNKZEEqDY2v+lkwKJUmSVHmnnnoqu+++O5MmTep3m8zkIx/5CPvuuy9Tpkxh8eLFG9ddc801TJw4kX333Zc5c+Y0dMxddtllm+PeWpdddhnLly8v7fhFaOQ9+N73vseUKVOYMmUKBx98ML/73e82rttnn32YPHkyBxxwAB0dfc7x/hyt8h6aFEqSJKmpzL99GdPm/IIJs3/KtDm/YP7ty7b5NU855RSuueaazW5z9dVX88ADD/DAAw8wd+5czjzzTAA2bNjAhz70Ia6++mruuecevv/973PPPfdsVRwbNmzYqv0GqvSkcMk8uGASnD269rhk3ja9XKPvwYQJE/jVr37FkiVL+PSnP83pp5++yfobbriBO+64g87Ozm2KZSiYFEqSJKmS5t++jE9ecSfLVq8hgWWr1/DJK+7c5sTwkEMO4QUveMFmt7nqqqt4z3veQ0Twute9jtWrV/Poo49y2223se+++/KSl7yEESNGcMIJJ3DVVVc9Z/+HHnqI17/+9Rx44IF8+tOf3lj+y1/+kkMPPZSTTjqJyZMn8/TTT/O+972PyZMnM3XqVG644QaglgRMnz6do446iokTJ/KZz3xm42t84QtfYNKkSUyaNIkvfvGLADz88MOb1Hyef/75nH322fzwhz+ks7OTd7/73RxwwAGsWbNmm353A7ZkHvz4I/DEUiBrjz/+yDYlho2+BwcffDC77bYbAK973evo6uoa0HFa9T109FFJkiQ1jfOuvY816zatiVmzbgPnXXsfx04dV+ixly1bxvjx4zcut7e3s2zZsj7Lb7311ufs/9GPfpQzzzyT97znPVx00UWbrLvtttu46667mDBhAp///OcBuPPOO/n973/PEUccwf3337/JdjvttBMHHnggb33rW4kIvvWtb3HrrbeSmbz2ta/ljW9848bkp7d3vOMdXHjhhZx//vkNN5McVNefA+t6JTHr1tTKt3Latkbfg54uueQSjj766I3LEcERRxxBRHDGGWc8pxYRWvc9tKZQkiRJTWP56r5rRPorH0yZ+ZyyiOi3vLebbrqJE088EYCTTz55k3UHHXTQxikEbrzxxo3r999/f1784hdvTCgOP/xwxowZQ1tbGzNmzODGG2/kxhtv5LjjjmPnnXdml112YcaMGfzmN7/ZtpMt0hP91M71V96ARt+DbjfccAOXXHIJn/3sZzeW3XTTTSxevJirr76aiy66iF//+tfP2a9V30OTwhIU0Q5ekiSpCvYa3Tag8sHU3t7O0qVLNy53dXWx11579Vvel/4SlZ133nnj874SnP727y8pBdhxxx155plnNi4PdJqCwoxqH1h5AwbyHixZsoT3v//9XHXVVYwZM2Zjeff2u+++O8cddxy33XZbn/u34ntoUjjEimoHL0mSVAWzjpxI2/Bhm5S1DR/GrCMnFn7sY445hu985ztkJrfccgujRo1izz335MADD+SBBx7goYceYu3atVx++eUcc8wxz9l/2rRpXH755UBtFMz+HHLIIRvX33///TzyyCNMnFg7v+uuu44///nPrFmzhvnz5zNt2jQOOeQQ5s+fz1NPPcWTTz7JlVdeyT/90z+xxx57sGLFClatWsU//vEPfvKTn2w8xq677srf/va3wfz1NO6ws2B4ryR+eFutfCs1+h488sgjzJgxg+9+97u87GUv21j+5JNPbvx9PPnkk/z85z/vcyTaVn0P7VM4xMpsBy9JktTsur8vnXftfSxfvYa9Rrcx68iJ2/w96sQTT+SXv/wljz/+OO3t7XzmM5/htNNO4+KLLwZg5syZvOUtb+FnP/sZ++67LzvttBPf+ta3gFptzoUXXsiRRx7Jhg0bOPXUU3nlK1/5nGN86Utf4qSTTuJLX/oSb3/72/uN5YMf/CAzZ85k8uTJ7Ljjjlx22WU873nPA+ANb3gDJ598Mg8++CAnnXTSxv5kp5xyCgcddBAA73//+5k6dSoAZ511Fq997WuZMGEC+++//8ZjnHLKKcycOZO2tjZ++9vf0tZWfE3rRt39Bq8/p9ZkdFR7LSHcyv6EsPn3oOd7eM4557Bq1So++MEPbtyvs7OTxx57jOOOOw6A9evXc9JJJ3HUUUc95zit+h7G5qo2W0VHR0duy7Cyg2nC7J/S1288gIfmvHWow5EkSSrdvffey8tf/vKyw9juXXbZZXR2dnLhhReWHYq20lC9h339TUXEoszsc1Qam48OsTLbwUuSJElSbyaFQ6zMdvCSJElqXqeccoq1hE1ue30P7VM4xIpqBy9JkiRJW8OksATHTh1nEihJktRDZm52XjlJjdmaMWNsPipJkqRSjRw5klWrVm3Vl1lJz8pMVq1axciRIwe0nzWFkiRJKlV7eztdXV2sXLmy7FCkpjdy5Eja29sHtI9JoSRJkko1fPhwJkyYUHYYUmXZfFSSJEmSKsykUJIkSZIqzKRQkiRJkiosqjDKU0SsBP6n7Dj68ELg8bKDUMvy+lKRvL5UNK8xFcnrS0XaXq+vF2fm2L5WVCIp3F5FRGdmdpQdh1qT15eK5PWlonmNqUheXypSM15fNh+VJEmSpAozKZQkSZKkCjMpLNfcsgNQS/P6UpG8vlQ0rzEVyetLRWq668s+hZIkSZJUYdYUSpIkSVKFmRSWICKOioj7IuLBiJhddjxqPRHxcETcGRF3RERn2fGouUXEpRGxIiLu6lH2goi4LiIeqD/uVmaMal79XF9nR8Sy+mfYHRHxljJjVPOKiPERcUNE3BsRd0fER+vlfoZpm23m+mq6zzCbjw6xiBgG3A8cDnQBC4ETM/OeUgNTS4mIh4GOzNwe58hRk4mIQ4C/A9/JzEn1ss8Bf87MOfWbW7tl5ifKjFPNqZ/r62zg75l5fpmxqflFxJ7Anpm5OCJ2BRYBxwKn4GeYttFmrq/jabLPMGsKh95BwIOZ+cfMXAtcDkwvOSZJ6ldm/hr4c6/i6cC368+/Te2foDRg/Vxf0qDIzEczc3H9+d+Ae4Fx+BmmQbCZ66vpmBQOvXHA0h7LXTTpxaPtWgI/j4hFEXF62cGoJe2RmY9C7Z8isHvJ8aj1fDgiltSbl9q0T9ssIvYBpgK34meYBlmv6wua7DPMpHDoRR9ltuHVYJuWma8GjgY+VG+eJUnN4mvAS4EDgEeBz5cbjppdROwC/Aj4WGb+tex41Fr6uL6a7jPMpHDodQHjeyy3A8tLikUtKjOX1x9XAFdSa7YsDabH6n0puvtUrCg5HrWQzHwsMzdk5jPAN/AzTNsgIoZT+8L+vcy8ol7sZ5gGRV/XVzN+hpkUDr2FwH4RMSEiRgAnAAtKjkktJCJ2rnd2JiJ2Bo4A7tr8XtKALQDeW3/+XuCqEmNRi+n+sl53HH6GaStFRACXAPdm5hd6rPIzTNusv+urGT/DHH20BPVhab8IDAMuzcz/V3JIaiER8RJqtYMAOwL/5TWmbRER3wfeBLwQeAz4N2A+MA/YG3gEeGdmOliIBqyf6+tN1JpdJfAwcEZ3/y9pICLiDcBvgDuBZ+rF/5davy8/w7RNNnN9nUiTfYaZFEqSJElShdl8VJIkSZIqzKRQkiRJkirMpFCSJEmSKsykUJIkSZIqzKRQkiRJkirMpFCSpAZFxIaIuKPHz+xBfO19ImK7n8tKktR6diw7AEmSmsiazDyg7CAkSRpM1hRKkrSNIuLhiPhsRNxW/9m3Xv7iiLg+IpbUH/eul+8REVdGxO/qPwfXX2pYRHwjIu6OiJ9HRFtpJyVJqgyTQkmSGtfWq/nou3qs+2tmHgRcCHyxXnYh8J3MnAJ8D/hyvfzLwK8y81XAq4G76+X7ARdl5iuB1cDbCz4fSZKIzCw7BkmSmkJE/D0zd+mj/GHgzZn5x4gYDvwpM8dExOPAnpm5rl7+aGa+MCJWAu2Z+Y8er7EPcF1m7ldf/gQwPDP/o/gzkyRVmTWFkiQNjuzneX/b9OUfPZ5vwL7/kqQhYFIoSdLgeFePx9/Wn98MnFB//m7gxvrz64EzASJiWEQ8f6iClCSpN+9ASpLUuLaIuKPH8jWZ2T0txfMi4lZqN1xPrJd9BLg0ImYBK4H31cs/CsyNiNOo1QieCTxaePSSJPXBPoWSJG2jep/Cjsx8vOxYJEkaKJuPSpIkSVKFWVMoSZIkSRVmTaEkSZIkVZhJoSRJkiRVmEmhJEmSJFWYSaEkSZIkVZhJoSRJkiRVmEmhJEmSJFXY/wfFQUpDyRHkWgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x1080 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout in dropout_choices:\n",
    "  solver = solvers[dropout]\n",
    "  train_accs.append(solver.train_acc_history[-1])\n",
    "  val_accs.append(solver.val_acc_history[-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].train_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].val_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 2:\n",
    "Compare the validation and training accuracies with and without dropout -- what do your results suggest about dropout as a regularizer?\n",
    "\n",
    "## Answer:\n",
    "[FILL THIS IN]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 3:\n",
    "Suppose we are training a deep fully-connected network for image classification, with dropout after hidden layers (parameterized by keep probability p). If we are concerned about overfitting, how should we modify p (if at all) when we decide to decrease the size of the hidden layers (that is, the number of nodes in each layer)?\n",
    "\n",
    "## Answer:\n",
    "[FILL THIS IN]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
